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DETAILED ACTION 

Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 

2. Claims 1 - 7 and 14 - 20 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Stauffer et al. ("Adaptive background mixture models for real-time 
tracking", Proceedings 1999 IEEE Conference on Computer Vision and Pattern 
Recognition. Fort Collins, Col., 1999 June 23 - 25; 2:246 - 252). 

3. In regards to claim 1 , Stauffer discloses a method for use in monitoring a search 
area, the method comprising (in the second paragraph of the abstract on page 246 a 
"stable, real-time outdoor tracker which reliably deals with lighting changes, repetitive 
motions from clutter, and long-term scene changes." Stauffer is disclosing a method for 
monitoring a search area, which must be preformed for tracking.): 

providing frames of image data representative of a search area, the image data 
comprising pixel value data for a plurality of pixels (In the second paragraph of the 
introduction on page 246 Stauffer discloses using the tracking method for video 
surveillance. Inherent in yideo surveillance is the step of providing frames of image 
data representative of the search area. As a computer must perform this method the 
image data must comprise of pixel value data.); 
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providing a plurality of time varying distributions for each pixel based on the pixel 
value data (Stauffer discloses providing a plurality of time varying distributions on page 

248 in the second column, second full paragraph. "The recent history of each pixel [...] 
is modeled by a mixture of K Gaussian distributions."); 

providing at least one frame of update image data representative of the search 
area in an update cycle, the frame of image data comprising update pixel value data for 
each of the plurality of pixels (As previously noted Stauffer's tracking method is 
designed to deal with "long-term scene changes" and thus every image frame can be 
considered an update image, the frame would comprise of update pixel value data for 
each of the plurality of pixels. Stauffer notes in the first partial paragraph on page 249. 
"so we use an approximate method which essentially treats each new observation as a 
sample set of size 1 and uses standard learning rules to integrate the new data."); and 

attempting to match the update pixel value data for each pixel to each of all of the 
plurality of time varying distributions provided for the pixel (In the first full paragraph on 
page 249 Stauffer discloses the step of attempting to match the update pixel data to the 
time varying distributions. "Every new pixel value, Xt, is checked against the existing K 
Gaussian distributions, until a match Is found."); 

updating the plurality of time varying distributions for each pixel based on 
whether the update pixel value data matches one of the plurality of time varying 
distributions provided for the pixel (Stauffer notes in the second full paragraph on page 

249 that "if none of the K distributions match the current pixel value, the least probable 
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distribution is replaced". This is the step of updating the plurality of time varying 
distributions based on whether there is a match or not.); and 

ordering the updated plurality of time varying distributions for each pixel based on 
a probability of the time varying distributions thereof being representative of background 
or foreground information in the search area for use in determining whether the pixel is 
to be considered background or foreground information (The second full paragraph on 
page 249 shows that the time varying distributions are ordered. Finally in the second 
paragraph of Background Model Estimation on the same page Stauffer discloses that 
the time varying distributions represent either background or foreground (new object). 
Stauffer further shows that background and foreground are differentiated in the first 
paragraph under Connected Components on page 250, "the method described above 
allows us to identify foreground pixels in each new frame".). 
4. In regards to claim 2, Stauffer discloses the method of claim 1 , wherein 
attempting to match the update pixel value data for each pixel to each of all of the 
plurality of time varying distributions provided for the pixel comprises: 

providing a narrow distribution for the pixel (In the first full paragraph on page 
249 that the matching step defines a narrow distribution); and 

comparing the narrow distribution to each of all of the plurality of time varying 
distributions provided for the pixel (Stauffer discloses in the first full paragraph on page 
249 that the matching step defines a narrow distribution ("a match is defined as a pixel 
value within 2.5 standard deviations of a distribution"), which is compared to each of a 
plurality of time varying distributions). 
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5. In regards to claim 3, Stauffer discloses in the first full paragraph on page 249 
that "a match is defined as a pixel value within 2.5 standard deviations of a distribution". 
Computing a divergence would be necessary to find a match in the method disclosed by 
Stauffer. 

6. In regards to claim 3, Stauffer discloses the method of claim 2, wherein 
comparing the narrow distribution to each of all of the plurality of time varying 
distributions provided for the pixel comprises computing divergence between the narrow 
distribution created for the pixel and each of all the plurality of time varying distributions 
provided for the pixel (in the first full paragraph on page 249 that "a match is defined as 
a pixel value within 2.5 standard deviations of a distribution". Computing a divergence 
would be necessary to find a match in the method disclosed by Stauffer,). 

7. In regards to claim 4, Stauffer discloses the method of claim 4, wherein updating 
the plurality of time varying distributions for each pixel comprises generating a pooled 
distribution based on the narrow distribution and a matched distribution if the narrow 
distribution matches one of the plurality of time varying distributions, and further wherein 
ordering the updated plurality of time varying distributions comprises determining if the 
pixel is representative of background or foreground information in the search area 
based on a position of the pooled distribution within the order of the updated plurality of 
time varying distributions (the majority of this claim has already been addressed above; 
the pooled distribution was shown in claim 1 . It has already been shown that Stauffer 
differentiates foreground and background. Stauffer in the first full paragraph in the 
second column of page 249 that while a pixel is in the foreground it will be at the front of 
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the distribution list, once it reverts back to background the foreground distribution will 
quickly fall off the list. Thus the list is ordered by the distinction between foreground and 
background.). 

8. In regards to claim 5, Stauffer discloses the method of claim 2, wherein updating 
the plurality of time varying distributions for each pixel comprises replacing one of the 
plurality of time varying distributions with a new distribution if the narrow distribution 
does not match one of the plurality of time varying distributions, and further wherein 
ordering the updated plurality of time varying distributions comprises assuring that the 
new distribution is representative of foreground information in the search area (the 
majority of this claim has already been addressed above, in particular the steps of 
replacing one of the distributions. As shown prior, Stauffer discloses in the first 
paragraph of the Connected Components section on page 250 that foreground pixels 
are identified.). 

9. In regards to claim 6, Stauffer discloses the method of claim 1 . wherein ordering 
the updated plurality of time varying distributions for each pixel is based on weights 
associated with the plurality of time varying distributions (Stauffer discloses in the third 
full paragraph on page 249 that the ordering of the plurality of time varying distributions 
for each pixel is based on weights associated with the plurality of time varying 

■X. 

distributions.). 

10. In regards to claim 7, Stauffer discloses the method of claim 1 , wherein at least a 
portion of the foreground information corresponds to one or more moving objects, and 
further wherein the method comprises tracking the one or more moving objects in the 
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search area to determine object paths for the one or more moving objects (Stauffer 
discloses in the two paragraphs under the Connected Components category on page 
250 that parts of the foreground are grouped into moving objects. Finally in the second 
paragraph under Multiple Hypothesis Tracking Stauffer discloses that multiple 
hypotheses tracking is used to track moving objects.). 

11. In regards to claim 14, Stauffer discloses a system for use in monitoring a search 
area, the system comprising (Stauffer discloses in the second paragraph of the abstract 
on page 246 a "stable, real-time outdoor tracker which reliably deals with lighting 
changes, repetitive motions from clutter, and long-term scene changes." Stauffer is 
disclosing a system for monitoring a search area, which must be preformed for 
tracking.): 

one or more imaging devices operable to provide frames of image data 
representative of the search area, the image data comprising pixel value data for a 
plurality of pixels, wherein the frames of image data comprises at least one frame of 
update image data representative of the search area in an update cycle, the frame of 
update image data comprising update pixel value data for each of the plurality of pixels 
(In the second paragraph of the introduction on page 246 Stauffer discloses using the 
tracking method for video surveillance. Video surveillance of course requires an imaging 
device. Inherent in video surveillance is the step of providing frames of image data 
representative of the search area. As a computer must perform this method the image 
data must comprise of pixel value data. Stauffer discloses providing a plurality of time 
varying distributions on page 248 in the second column, second full paragraph. "The 
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recent history of each pixel [...] is modeled by a mixture of K Gaussian distributions."); 
and 

a computer apparatus operable to (A computer of some sort is necessary to carry 
out the operations disclosed by Stauffer): 

attempt to match the update pixel value data for each pixel to each of all of the 
plurality of time varying distributions provided for the pixel (In the first full paragraph on 
page 249 Stauffer discloses the step of attempting to match the update pixel data to the 
time varying distributions. "Every new pixel value, Xt, is checked against the existing K 
Gaussian distributions, until a match is found." Stauffer notes in the next paragraph that 
"if none of the K distributions match the current pixel value, the least probable 
distribution is replaced". This is the step of updating the plurality of time varying 
distributions based on whether there is a match or not.); 

update the plurality of time varying distributions for each pixel based on whether 
the update pixel value data matches one of the plurality of time varying distributions 
provided for the pixel (As previously noted Stauffer's tracking method is designed to 
deal with "long-term scene changes" and thus every image frame can be considered an 
update image, the frame would comprise of update pixel value data for each of the 
plurality of pixels. Stauffer notes in the first partial paragraph on page 249, "so we use 
an approximate method which essentially treats each new observation as a sample set 
of size 1 and uses standard learning rules to integrate the new data."); and 

order the updated plurality of time varying distributions for each pixel based on a 
probability of the time varying distributions thereof being representative of background 
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or foreground information in the search area for use in determining whether the pixel is 
to be considered background or foreground information (The second full paragraph of 
page 249 shows that the time varying distributions are ordered. Finally in the second 
paragraph of Background Model Estimation on the same page Stauffer discloses that 
the time varying distributions represent either background or foreground (new object). 
Stauffer further shows that background and foreground are differentiated in the first 
paragraph under Connected Components on page 250, "the method described above 
allows us to identify foreground pixels in each new frame".). 

12. In regards to claim 15,Staiffer discloses the system of claim 14, wherein the 
computer apparatus is further operable, with respect to each pixel, to (); 

provide a narrow distribution for the pixel (In the first full paragraph on page 249 
that the matching step defines a narrow distribution); and 

compare the narrow distribution to each of all of the plurality of time varying 
distributions provided for the pixel (Stauffer discloses in the first full paragraph on page 
249 that the matching step defines a narrow distribution ("a match is defined as a pixel 
value within 2.5 standard deviations of a distribution"), which is compared to each of a 
plurality of time varying distributions). 

13. In regards to claim 16, Stauffer discloses the system of claim 14, wherein the 
computer apparatus is further operable, with respect to each pixel, to compute 
divergence between the narrow distribution provided for the pixel and each of all the 
plurality of time varying distributions provided for the pixel (Stauffer discloses in the first 
full paragraph on page 249 that "a match is defined as a pixel value within 2.5 standard 
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deviations of a distribution". Computing a divergence v\^ould be necessary to find a 
match in the method disclosed by Stauffer.). 

14. In regards to claim 17, Stauffer discloses the system of claim 15. wherein the 
computer apparatus is further operable, with respect to each pixel, to: 

update the plurality of time varying distributions by generating a pooled 
distribution based on the narrow distribution and a matched distribution if the narrow 
distribution matches one of the plurality of time varying distributions (the majority of this 
claim has already been addressed above; the pooled distribution was shown in claim 
16.); and 

determine if the pixel is representative of background or foreground information 
in the search area based on position of the pooled distribution within the order of the 
updated plurality of time varying distributions (It has already been shown that Stauffer 
differentiates foreground and background. Stauffer in the first full paragraph in the 
second column of page 249 that while a pixel is in the foreground it will be at the front of 
the distribution list, once it reverts back to background the foreground distribution will 
quickly fall off the list. Thus the list is ordered by the distinction between foreground and 
background.). 

1 5. In regards to claim 1 8, Stauffer discloses the system of claim 1 5, wherein the 
computer apparatus is further operable, with respect to each pixel, to: 

update the plurality of time varying distributions by replacing one of the plurality 
of time varying distributions with a new distribution if the narrow distribution does not 
match one of the plurality of time varying distributions (the majority of this claim has 
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already been addressed above, in particular the steps of replacing one of the 
distributions); and 

assure that the new distribution is representative of foreground information in the 
search area (As shown prior, Stauffer discloses in the first paragraph of the Connected 
Components section on page 250 that foreground pixels are identified.). 

16. In regards to claim 19. Stauffer discloses the system of claim 14, wherein the 
computer apparatus is further operable to order the updated plurality of time varying 
distributions for each pixel based on weights associated with the plurality of time varying 
distributions (Stauffer discloses in the third full paragraph on page 249 that the ordering 
of the plurality of time varying distributions for each pixel is based on weights associated 
with the plurality of time varying distributions.). 

17. In regards to claim 20, Stauffer discloses the system of claim 14, wherein at least 
a portion of the foreground information corresponds to one or more moving objects, and 
further wherein the computer apparatus is operable to track the one ore more moving 
objects in the search area to determine object paths for the one or more moving objects 
(Stauffer discloses in the two paragraphs under the Connected Components category 
on page 250 that parts of the foreground are grouped into moving objects. Finally in the 
second paragraph under Multiple Hypothesis Tracking Stauffer discloses that multiple 
hypotheses tracking is used to track moving objects.). 

Claim Rejections - 35 USC § 103 

18. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

19. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

20. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

21. Claims 8-10 and 21 - 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Stauffer in view of Sacks (4,739,401). 

22. In regards to claims 8 and 21 , Stauffer discloses in the first paragraph in the 
Connected Components section segmenting (calculating blobs) the foreground with the 
"connected components algorithm". Stauffer, however, does not disclose filtering out 
blobs have less than a predetermined pixel area size. 
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23. Sacks teaches in the paragraph starting on column 4, line 29 that size can be 
used to filter out "objects from the image being processed which are not potential 
targets". "The size identification subsystem provides second output signals indicative of 
objects located within the image scene whose sizes are within a predetermined size 
range." Thus Sacks teaches that blobs having less than a predetermined pixel area size 
are filtered out. 

24. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to use size to filter out objects that are two small, as taught by 
Sacks, before performing the tracking operations as disclosed by Stauffer. If an outdoor 
tracking system is intended to track cars or people having a size cut off for objects of 
interest makes sense as small animals, leaves, and other small objects would be 
tracked without the cut off, leading to wasted processing time. 

25. In regards to claims 9, 10, 22, and 23, Stauffer discloses in the second 
paragraph in the Multiple Hypotheses Tracking section on page 250 that connected 
components are grouped into object paths (this is the correspondence between frames 
Stauffer writes about in the paragraph) using a multiple hypotheses tracking algorithm. 

26. Claims 11,12, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Stauffer in view of Baxter (5,966,074). 

27. In regards to claim 1 1 , Stauffer teaches of using the tracking method disclosed 
for tracking objects in an outdoor environment as well as classifying those objects 
(people and cars), however Stauffer does not disclose classifying object paths as 
normal or abnormal. 
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28. Baxter teaches in the paragraph starting at column 7, line 65 that object path 
models can be created based on position, trajectory, angle, and speed and used to 
identify an object path as either normal or abnormal (sets off an alarm). An object path 
is then compared to these models to classify that object path. 

29. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to add a path classifying module as taught by Baxter to the tracking 
method disclosed by Stauffer. Being able to classify a path as normal or abnormal 
would be useful to the tracking method, as it would allow for a complete surveillance 
system. For example if the method is used to track cars with the path classification 
added in the method will be able to identify speeding cars for ticketing. 

30. In regards to claim 12, Stauffer in view of Baxter discloses the method of claim 

1 1 , wherein providing one or more defined normal and/or abnormal object path feature 
models comprises providing one or more defined threatening and/or non-threatening 
object path feature models based on one or more characteristics associated with 
threatening events (threatening paths and non-threatening paths can be correlated to 
abnormal/normal paths respectively.); and 

wherein comparing the one or more object paths to the one or more defined 
normal and/or abnormal object path feature models comprises comparing at least the 
one or more object path, or data associated therewith, to the one or more defined 
threatening and/or non-threatening object path feature models to determine whether the 
one or more object paths appear to indicate that a threatening event is occurring (One 
threatening characteristic disclosed by Baxter as shown above in the rejection of claim 
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1 1 is speeding. Again as shown by Baxter if a threatening object path is detected an 
alarm is activated. The rest of this claim is addressed in the rejection of claim 11.). 

31 . In regards to claims 24 and 25, Stauffer teaches of using the tracking method 
disclosed for tracking objects in an outdoor environment as well as classifying those 
objects (people and cars), however Stauffer does not disclose classifying object paths 
as normal or abnormal. 

32. Baxter teaches in the paragraph starting at column 7. line 65 that object path 
models can be created based on position, trajectory, angle, and speed and used to 
identify an object path as either normal or abnormal (sets off an alarm). An object path 
is then compared to these models to classify that object path. 

33. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to add a path classifying module as taught by Baxter to the tracking 
system disclosed by Stauffer. Being able to classify a path as normal or abnormal(non- 
threatening or threatening) would be useful to the tracking method, as it would allow for 
a complete surveillance system. For example if the system is used to track cars with 
the path classification added in the method will be able to identify speeding cars for 
ticketing. 

34. Claims 13 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Stauffer in view of Uyttendaele. 

35. In regards to claims 13 and 26, Stauffer discloses a method and system for 
tracking objects. Stauffer however, does not discuss combining multiple imaging 
devices into one overlapping image for use in tracking. 
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36. Uyttendaele discloses In figure 3A a method for combining a plurality of frames of 
image pixel data. In step 300 Uyttendaele acquires multiple frames of image pixel data. 
In the paragraph starting at column 4, line 45 Uyttendaele discloses, "a camera 163 
capable of capturing a sequence of images 164 can also be included as an input device 
to a personal computer." Any image inputted into a computer must be comprised of 
pixels. Uyttendaele then discloses in the paragraph starting at column 5, line 50 that 
each frame is "captured by a different cameras from a different viewpoint." Uyttendaele 
discloses in the paragraph starting at column 7, line 41 that "the lateral field of view of 
each camera overlaps by at least 20 percent." This is about 25 percent. As each image 
would overlap with two other images the total overlap for any one image would be about 
40 percent. This is less than 85 percent of the field of view. Returning to figure 3A 
Uyttendaele discloses in step 306 that the image frames are combined (mosaic). 

37. Therefore it would have been obvious to one having ordinary skill in the art at the 
time of the invention to use multiple overlapping cameras as taught by Uyttendaele to 
obtain a complete search area at high resolution for tracking as disclosed by Stauffer. 
By using multiple cameras a higher resolution image can be obtained which allows for 
more detail and thus more accurate tracking. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Christopher L Lavin whose telephone number is 703- 
306-4220. The examiner can normally be reached on M - F (8:30 - 5:00). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on (703) 305-4706. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status infomnation for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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